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APPENDIX A 

The Belmont Agreement. 

Cooperative Arrangement for Evaluation of 

Elementary & Secondary School Programs 

In recognition of their common concern for effective evaluation 
of elementary and secondary education programs in the United Stales, 
the Council of Chief Stale School Officeis and the Commissioner of 
Education of the United States ngrec*. 

I. GENERAL : 

a. Within the context of present Federal programs, 

Jointly to develop and install a common survey 
instrument that is designed to meet the basic and 

. common management requirements of the Office of 

Education and State Education Agencies for evaluation 
of elementary and secondary education programs; 

b. Jointly, to develop and install pilot training 
programs for evaluation personnel in State and local 
education agencies; and 

c. Jointly, to develop and implement a long-range pro- 
gram of general and evaluative information for 
elementary and secondary education in the United 
States. 

11 . ^YPES OF DATA : 

The common survey instrument would be used to aid in the 
evaluation of all or portions of the following Federal 
programs: ESF,A I, II, 111, V, VI, VII, VIII: NDEA III 

and V-A, and relevant parts of Vocational Education. 

The instrument would be confined to those principal 
classes of information that are deemed to bo crucial to 
the management interests both of Stale Education Agencies 
and the Office of Education. Reasonable standards of 
validity and reliability will govern the choice of 
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, specific items to be included. 

III. PARTI CIPATIO N: 

It is understood that in FY 69 only a broadly represen- 
tative group of Statcs--Miilunlly agreed upon--will 
participate in a pilot instrument development and 
implementation effort. 

IV. USES OF DATA : 

Data produced under this agreement will be available 
freely to the participating States and the Office of 
Education. 

It is understood that the results of any "anchor" 
tests that may be developed will be oppliod by the Of- 
fice of Education in those States mutually agreed upon 
by a joint OE/CSSO policy level task force. 

V . TKPLEhT. STATIO N : 

The instrument development and pilot program will 
proceed as rapidly as possible. 

V I . PARTICIPATION Or LK As IN PILOT PROGRAM : 

It is understood that each participating State will 
arrange for representative LEAs to become involved in 
the instrument development and pilot programs. 

AUGUST 29, 1968 
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The Belmont States and their Representation 

The Twenty States Included in the Belmont Croup Are*. 

KBPRBSKKTAT1 VBS 



1. 


Washington 


1 


2. 


California 


1 


3. 


Texas 


1 


4. 


Colorado 


1 


5. 


North Dakota 


1 


6. 


South Dakota 


X 


7. 


Minnesota 


2 


8 . 


Wisconsin 


1 


9. 


Michigan 


l 


10. 


Illinois (inactive) 
Replaced by Pennsylvania 


3 


11. 


Kentucky 


1 


12. 


Ohio 


1 


13. 


New York 


2 


14. 


New Jersey 


2 


15. 


Connecticut 


1 


16. 


Georgia 


t 




17 



Florida 



1 
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18. 


Maryland 


2 


19. 


North Carolina 


2 


20. 


South Carol Ina 


1 



1c ☆ & if Is *Sr £ 

There are 17 representatives from the U.S. Office of Educa- 
tion to the Belmont Group. 
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SUMMARY OF LEGISLATIVE TITLES COVERED 

Federal support for education takes place under a number of 
legislative Title??. The substance of each of the Titles with which 
the JCES is presently concerned is presented in this Appendix. The 
purpose of this summary is to give some indication of the types of 
,_educaLlonal programs and services that are to be evaluated by the JCES, 

The extractions below were taken frorn B ureau of Elemen tar y and 

Secondary E duc ation Program s, U.S.O.E., March 1969, and from the lcgisln- 

tive Titles themselves. The names of the Titles that are summarized 

♦ 

are as follows: 

1. ESEA of 1965, Title I*. Special Programs for Educationally 
Deprived Children 

2. ESEA of 1965, Title II: School Library Resources, Textbooks, 

and other Instructional Materials 

3. ESEA of 1965, Title III: Supplementary Educational Centers 

and Services 

A. ESEA of 1965, Title V: Grants to Strengthen State Departments * 

of Education 

5. ESEA of 1965, Title VII: Bilingual Education Programs 

6. ESEA of 1965, Title VI t 1 : Dropout Prevention Program 

7. NDFA of 1958, Title III: Strengthening Instruction in Critical 

Subjects 
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• 8. NDEA of 1958, Title V-A: Counseling, Guidance and Testing; 

Identification and Encouragement of Able Students 

9. Civil Rights Act of 1964, Title IV: Equal Educational Oppor- 

tunities 

10. Follow Through (Port of FSiA, 1965, Title 1) 

11. Vocational Education Amendments of 1968. 

I . KSKA of 1965, Title I: Special Programs for Educali onal ly Deprived 

Children 

This title offers financial assistance to State end local 
educational agencies to expand and improve their educational programs 
for: 

•• educationally disodvantaged children in low-income areas 

- handicapped, neglected, delinquent, and foster chile on 

• children of migratory agricultural workers 

- American Indian children attending Bureau of Indian Affairs 
schools 

Programs are designed to give special educational assistance 
to children whoso educational levels “*‘e below normal. Projects may 
provide such services as supplementary and remedial Instruction in 
reading and mathematics, pupil and family counseling, cultural enrichment, 
and pre-school activities. 

Approximately 9.2 million disadvantaged children in 16,000 
school districts participated during F\68. h total of $1.12 Billion 
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vas allocated for 1Y69 programs. 

2 . I'JSEA of 1963, Tf ll c II: S choo l bl hrniy Hc r.onrcco, To; 1 books, an d 

other Tn sinicLion.il Material s 

The purpose of this Title is to improve instruction by providing 
funds to States for school library resources, textbooks, and other 
printed and published instructional materials for the use of children 
and teachers in public and private elementary and secondary schools. 

Funds may be used to purchase, books, periodicals, documents, pamphlets, 
photographs, reproductions, pictorial or graphic vorks, musical scores, 
maps, charts, globes, sound recordings, processed slides, transparencies, 
films, filmstrips, kinescopes, and video tapes. Excluded are equipment 
and furniture, materials intended for religious instruction, tnd materials 
consumed in use or those which cannot be expected to last for more than 
one year. 

Almost 44.6 million children and 1,8 million teachers 
participated in this program in 1969. For IY68 and FY69, $99 million 
and $50 million, respectively, were made available under this program. 

3 • F SEA of 1965, Title I. II? Sup p 1 ctrentary IkHicaUonal Center and Ser v ices 

The purpose of this program is to stimulate school dlsMicts to 
seek creative solutions to their educational problems. This program 
Is also known as PACK (Projects to Advance Creativity in ‘'ducat ion). 

Funds may be requested for innovative and exemplary applications of new 
educational knowledge or for vitally needed supplementary services. 





The following types of projects are funded: improved programs 

for deprived inner city children; programs of individualized instruct ion 
for self-paced learning; early childhood educational programs; program 
of quality education for minority group children and children in 
geographically isolated areas; special programs for handicapped children; 
development of planning and evaluation competence. 

A total of $165 million was allocated in FY 69. There 
are approximately 1800 projects under this Title which roach 10 million 
school pupils, 135,00 pre-school children, and 67,000 out of school 
youths. 

4 . KS EA of 1965, T i tle V: G r ants to Streng t hen State Departments of 

Kduca t i on 

The purpose of this Title is to aid the State educational 
agencies in establishing and improving programs to identify and meet 
their educational needs. The following types of programs are funded: . 

planning and evaluation; data collection and processing; dissemination 
of information on the condition, progress, and needs of education; 
research programs; distribution of new curriculum materials; teacher 
training; use of teacher aids; financing of education; measurement of 
pupil achievement; improvement of personnel in leadership, administration, 
and specialist services; consultative and technical services; use of 
Head Start benefits; comprehensive planning. 
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$29, 750,000 was appropriated f or FY63 and the same amount for 

FY69 . 

5 • ESEA of 1965, Ti tle VII: Bi 3 iiy.ua J. Education Programs 

This Title provides funds for educational programs for children 
3 to 18 years of age who have limited English-speaking ability and who 
come from non- English speaking environments. Funding is provided for 
research projects, development of instructional materials and training 
of teachers. Funding is also provided for the purchase of teaching 
materials and equipment. 

$7.5 million was appropriated for this program in FY69 , 

6 . JESEA of 1965, T it le VIII: Dropout Prevent io n Program 

The purpose of this Title is to develop programs for reducing 
the number of students who drop out of school. Programs may use innova- 
tive methods, systems, or materials which show promise in reducing 
dropouts . 

$5 million was allocated in FY69. 

7 . NDEA of 1958 , Title III: Strengthening Instruction in Criti.cal 

Subjects 

The purpose of this Title is to improve instruction in science, 
math, foreign language, social studies, economics, English, reading 
and industrial arts. Funds are available for purchase of laboratory 
and special equipment, printed materials other than textbooks, and 
administrative services in the designated subject areas. 
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In FY69, the* appropriations totaled over $78 million for this 

program, 

8 . NDKA of 1958, T itle V- A: Cou nseling Guidance and Tost fjvgj 

I dentificat i on and Enc o u r a v e m o n l of Able Students 

The purpose of tliis program is to provide funds for establish- 
ing guidance, counseling, and testing programs at elementary and secondary 
schools, junior colleges, and technical institutes. The funds are used 
to provide counseling and guidance personnel who are able to assess the 
abilities of students at an early age, assist students in selecting 
appropriate courses of study, apprise students of various training re- 
quirements and opportunities, and encourage them to continue their 
education, Funds are also used for programs of pupil personnel services 
where counselors work with social workers and psychologists to meet the 
needs of students and to recommend special instructional programs, 

The money appropriated for Title V-A was $24 , 5 million and 
$17 million for FY68 and FY69 respectively, 

9 , Civil Rights Act of 1964 , IV: Equal Educational Oppo rtunities 

The aim of this Title is to assist public school systems cope 
with school desegregation problems. This is done by grants to school 
boards for in-service and/or advisory specialist programs, contracts 
with universities for institutes to improve teacher ability, and techni- 
cal assistance to State education departments. 

Approximately 10,000 school personnel took part in 90 training 




programs during FY68. The appropriation for that year was $10 million. 

i 

10 . Foil ovy- T h roup h (Part of Tit le I, BS KA) 

Follow Through is designed to reinforce in the early grades the 
progress made by children in Head Start and other pre-school programs. 

The program is designed to meet the instructional, physical, and psycho- 
social needs of disadvantaged children in a program that combines 
school and community resources. Special classroom activities and addi- 
tion staff help meet instructional needs. Nutrition and health care 
are provided along with social work and psychological services, 

.The allocations for FY68 and FY69 were $15 million and $30 million 
respectively. During the 1968-1969 school year, about 91 projects served 
23,600 children, Forty- five new projects were funded in 1969-70. 

1 1 ' Voca t ional Ed uc ation Amendments of 1968 

The purpose of these amendments is to provide funds to States to 
help them maintain, extend, and improve existing programs of vocational 
education, develop new programs of vocational education, and provide 
part-time employment for youths who need earnings in order to continue 
their vocational training, More specifically, funds are granted to 
States (a) for conducting vocational education programs for persons of 
all ages; (b) for conducting vocational education for the handicapped; 

(c) for research in vocational education; (d) for development of new 
curricula; (c) for residential vocational education; (f) for programs in 



consumer and hoinemaking education; and (g) for cooperative work-study 



programs through local educational agencies and public and private 
employers . 
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Further Consideration of Evaluation Questions 

Assuming that the question of objectives has been satisfactorily 
resolved, evaluation may be approached from the point of view of 
programmatic description and also impact. Both of these types of evalu- 
ation may also be examined on a multi-level basis. In the first instance, 
the questions to be asked have to do with the nature and description of 
the program as implemented, that is, "What is actually being done?" 

"How does what is actually being done compared to what management intended 
to be done ? 11 "How many people (districts, communi Liles, items, objects, 
etc.) are being reached by the program ? 11 "Hot? many of what kinds of 
persons are involved in the implementation of the program?" • "What do 
various aspects of the program co*t? n Etc. 

The second kind of evaluation is more concerned with the measurable 
impacts of the program (whatever they are, and however the program may 
differ from the planned) on target groups (and others ). It asked questions 
such as "Can first grade children in this program display superior word 
attack skills as compared to those not in the program?" "Do children 
exposed to the program tend to be less prone to drop out than those 
not exposed?" "Is the attendance of program children better than that of 
those not in the program?" "Does the program result in n neglect on 
the part of teachers of some aspect of the curriculum not covered by 
the program?" Etc, 
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In addition to the above kinds of evaluation, it is sometimes 
useful to consider a certain kind of outcome of the program as a 
separate for evaluation purposes. This is the political and policy 
Impact of the program and/or its exi stance on various groups such as 
community lenders, board members, parents, etc. Strictly speaking, such 
outcome may be classed under impacts of the program, but it is sometimes 
appropriate to treat then separately from the targeted educational 
improvement objectives. 

The multi-level concept recognises that Federal programs are rarely 
unidimensional in design or implementation and that the educational 
establishment is basically hierarchical in nature. Thus, the institution 
of guidelines for evaluation at the LEA level also has implications and 
impact at the SEA level, and, for different reasons, at the Federal level 
as. veil. Therefore, the evaluation questions must specify the levels 
at v;hich they are concerned, and often be applied at more than one 
level . 

In the list of evaluation questions to follow, an effort has been 
made to keep them at a general enough level so that they may be general- 
ized across programs, legislative titles, and projects. They are divide 
into the three basic types mentioned above, and an indication of level 
is given. It should be recognized that the usefulness of the following 
list is confined to the examination of the components of the BCES to 
determine whether or not there are included within these components 



the basic items; of information which would permit relevant evaluation 



studies. To be useful fo** study designs and the construction of 
measurement instruments, a much finer level of detail will be required. 
Programion tic Deser t p tor 

1. What is the nature of the program? What projects are included? 
Subprojects? 

2. What are the purposes and objectives of the program as seen by 
those implementing it? 

3. VJhat portions of the program are and are not implemented? 

4* Who has the administrative responsibility for each part? 

5. What are the reasons that any planned implementation has not 
taken place? 

6. What are the effective (strong) points of the program? 

The ineffective (v:eak) points? (As seen by those implementing it). 

7* How do each of the above six questions compare to the planned 
implementation of the program? 

8. Who (what groups) are the direct participants in the program 
(at each level)? 

9* Who (what groups) are those indirectly impacted by the program? 

In what ways? 

10. How many (and what percentage) in each group defined are directly 
and indirectly impacted by the program? 
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5. Is target group performance different from that of similar, 
non- participating groups in more than one way? If so, how are these 
differences interrelated? 

6. In what ways arc the changes or differences characteristic of 
the target groups related to any or any combination of the program 
descriptor questions listed above? 

7* Are there any changes or differences with respect to the target 
group on non-performance variables such as attitudes, attendance, 
dropouts, etc. 

8. Are these interrelated among themselves or the descriptor 
ques tions? 

9* Has non- target but directly related educational group performance, 
attitudes, attendance, and the like changed or shown differences (teachers, 
counselors , non- Instructional personnel, para- profess Iona Is, administrative 
personnel, and the like)? 

/ 

10. What are the interrelationships of the various variable for 
these groups? 

Political/Policy Impac t 

1. Has the program received favorable or unfavorable publicity in 
the comrani ty? 

2. What is the community attitude toward the need versus the program? 

3. How do the parents feel toward the program? 
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4. Docs the program meet needs expressed by community groups? By 
parents? By school board members? By state and local officials? 

5* Does the program interface with other programs and projects 
currently receiving the support of local groups and agencies? 

* >V>V& *** * 

It should be noted that the above list of questions represents 
a first effort to take the concerns of evaluation to a greater degree 
of refinement. Further efforts along these lines are plannc-d for later 
phases of the project. Eventually, these refined evaluation questions 
will be cross-referenced against the components of the J5CES at the 
specific data level, and a determination will be made of the specific 
data lacks which may exist in the system. 
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APPENDIX K 



Descriptions of the following JCES instruments are presented in 
this appendix: 

1. Universe File 

2. ELSEGIS 

3. Program Reference File 

4. Consolidated Program Information Report 

5. SEA-MKS 

6. Pupil Centered Instrument 

7. Project Descriptor Questionnaire 

8. AIR Project Evaluation Guide 

9 . Common Status Measures 

The summaries presented below are based on detailed reviews of 
the JCES elements accomplished as part of the basis for a system analytic 
study of the developing JCES. The summaries were obtained by a close 
examination of the instruments themselves together with information ob- 
tained during talks with Office of Education staff members. Along with 
a general discussion of each instrument, a table giving an Item by item 
description of the instrument content is presented. 

1 . Universe File 

The National Center for Educational Statistics sends out once a 
year a brief questionnaire designed to gather certain items of very 
basic information about the schools and the school districts in the 
United States. The NCES Universe File is conceived as a basic sampling 
frame listing all schools by district within the states and regions 
of the country. It is intended that this file covci* both public and 
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private school, 0 . ?.nd be the most up to date, appropriate, and complete 
listing of schools and districts in the country. 

The kinds of informal: ion collected by the Universe File are as 
follows: a) Identification information Including name of district, 

name of school, name of county, mailing address, area codes, telephone 
numbers, etc,; b) type of school in terms of private, public, and grade 
span; c) population served in terms of number of students by grade and 
number of graduates during the preceding school year; d) assignment; 
e) program information (proposed for 1969-70) in terms of whether or 
not each of f:‘ve special programs are operating in the schools with or 
without Federal funding: handicapped, academically talented, compensatory, 

vocation/tcchnical, and continuing education. 

The following is an item by i tem description of the Universe File: 

1. School status: closed or in operation. 

2. USOE school code. 

3. State's School code. 

4« Name of School District. 

i 

5, Name and address of school. 

6, Name of County where school is located, 

7, Type of school: Elem, Middle, Sec*, Voc/Tcch, Area Voc* 

8, State or Federally Operated School? 

9, Number of pup! U by grade, 

10. Number of Graduates from Grade 12, 

11. Number of teaching and Non- Teaching Professional Personnel 
at each of the following levels: 

Pre- kindergarten 
Kindergarten 
Grad ed Elomen t a ry 
Secondary 
Special Education 
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* 12. Number of special programs operating and number operating 

with federal funds for each of the following groups: 
Handicapped 
Academically Talented 
Compensatory 
Vocationa 1 /Technical 
Conti nul ng Education 

El c men t ary and Se condar y Ge ner al Info r mation SysJ. cn (IL SfCTS ) 

The ELSEGIS Program is conducted by the National Center of Fduentfona 
Statistics and Is intended to he a general purpose survey of basic data 
regarding public school systems. For sone time, the emphasis In data 
collection by the National Center for Educational Statistics has been on 
the legislative requirements of programs such ns the NDEA and the ESEA 
Acts. However, NOES felt a need to collect data on school systems in 
theiv entirety rather than solely in terms of pvogrrn segments and a 
need to emphasise hasic data in response to the general educational 
community needs. Therefore, tVic National Center for Educational Statis- 
tics and the Committee of Educational Data Systems of the Council of 
Chief State School Officers developed an approach to collection of 
general survey data for public schools. It must be understood that 
strictly speaking, the EESEGIS effort is not a part of JCES, but the 
basic datfl which are collected on school systems seem to overlap consider- 
ably vith data to he collected through the Program Reference File, and 
with data to he collected by the CF1R. The basic data tables contained 
in the last M.SEGlS report (1967) are as follows: 1) the number of 

schools in local public school systems by organisational level, grade 



6pan; size of system; 2) enrollment and pupil teacher ratio in local 
public sysLcni by organizational level and size of school system; 3) 
instructional staff in local public school systems by position and size of 
system; A) classroom teachers by highest level of education completed 
and size of school system; 5) current expenditures of local public 
system by account and size of school sjsten; 6) current expenditures for 
pupil and average daily membership for local public system by account 
and size of system. 

It is not our purpose here to examine in detail the extent to 
which there may be duplication of effort in the various data collection 
efforts associated with CPIR program efforts file and ELSEGIS. It should 
simply be noLed that it will be our purpose at a later point in the 
present effort to take a critical look at the extent to which some of 
the information presently obtained by different forms can be combined in 
a single data collection effort. Thus, we may hold out the possibility 
that some aspect if not all of the ELSEGIS effort might bo merged with 
CPIR, or vice versa. 

In summary, the products to be expected out of the CPIR Instrument, 
in addition to its effectiveness in reducing the lead on local educational 
agencies in meeting the reporting requ l resents, are reports covering 
dollars expended by sources of funding; services and programs provided 
by these funds; identification of the number of children by target 



group needing services and t lie number of vices provided; in-service 
education by Federal source of funding, Tlic instrument will for Lhe 
first time provide a coordinated look with common descriptors a t the 
various Federal funding programs impacting on local school districts! 
end will enable interested parties to trace the flow of dollars by 
Federal sources to the kinds of programs and services provided with 
respect to the kinds of children benefited. 

The following is an item by item description of the KLSEGIS question- 
naire : 

1. Nano and address of public school system. 

?. Number of elementary and secondary schools by grade span. 

3. Number of pupils cntollcd in district in: 

Nursery 

Kindergarten 

Klomcnlr.ry 

Secondary 

4. Number of Instructional staff by type 

5. Number of teachers by level of education completed 



6. 


Average daily 


Attendance 


7. 


Average daily membership 


8. 


Yearly expenditures for 




6 . 


administration 




b. 


teacher salaries 




c. 


attendance services 




d. 


health services 




e. 


pupil transportation services 




f. 


plant operation 




8- 


plant maintenance 




h. 


food services 




1 . 


Student body activities 




J. 


community services 




3. P rogram Reference, Nile 

The concept of the Program Reference File is built on the cur- 
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rcntly existing National Center for Educational Statistics Universe File. 
See page E~1 of this Appendix. In order to be able to set up the kind 
of stratified sampling called for in employing the other measures in the 
Comprehensive Evaluation System, JClvS personnel would like to odd two 
questions to the NCES questionnaire. These two <- nest ions concern ur- 
banization and a detailed inquiry of the nature and kinds of Federal 
programs to be found in the schools. 

Mien FP&K analyzed the NCES Universe File, they quickly decided that 
the addition of these two items would be essential in order to provide 
the specific information concerning all different programs and projects 
funded by OK at various grade levels, and the population size from 
which the school draws Its students. At the present time, however, 
there seems to be some question os to whether or not it is going to prove 
feasible to add these two additional questions to the proposed 1070 
NCES questionnaire. If NCLS were to add the items to their questionnaire, 
it would ptobably be necessary to print two variations, one for the F.el- 
taont schools (for program reference purposes) and one for the remaining 
schools. For this reason, two forms might have to be mailed out. And 
for this reason, NCES is hesitant to add the two questions. If NCES is 
unwilling to add these two questions to their form, it is likely that 
F.CES will proetde to send out its own questionnaire, duplicating much 
of the information on the NCES questionnaire, but. adding the two questions 
referred to. 
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It is seen ns essential that ns this Program Reference Pile Is 
developed, there also be developed sample generating procedures useful 
at both federal and State levels. Additionally, os the Program Reference 
File is phased into the total project, It is essential that it become 
the single base for generating the samples to he used in collecting 
data for both the common management information report (district instru- 
ment) and the comprehensive evaluation system. To do otherwise negates 
the ability to interface data from both sources. 

Individual States may choose to add additional items to their Program 
Reference File to meet their needs ot the Stale level. 

The technical Task Force, composed of both Federal and State 
representatives, has assumed responsibility for the development of the 
instrument to collect information for the Program Reference File and the 
software for a sample generating system. Development may be accomplished 
on contract, supported by USOE, The availability of required data 
vill he determined in cooperation with participating Stale representatives. 
States will he responsible for conducting reviews with their local 
education agencies. The stales will be responsible for data collection 
within their respective States each Fall. This will entail mailing of 
forms, verification of receipt of forms in the local agencies, assuring 
that all local agencies respond, and initial editing of the returns 
within their respective States. The Office of Education will be 
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responsible for processing; of data and genera 1 i ng State-representative 
samples, subject to review and approval by each participating Stale. 

Completion of tt.e Program Reference Kile will allow the participating 
Stales to have a complete accounting of the projects operating in schools 
throughout their States. For example, Stntcs will be able to determine 
(1) the percentage and number of projects funded through KSKA, Title I 
which provide instruction in 'reading, arithmetic, science, etc., by 
grade level, (2) the number and proportion of elementary school which 
h8ve projects funded through ESMA, Title I and projects funded through 
KSEA, Title III, (3) the number and proportion of f-chools which have 
both Federally supported rending and arithmetic projects in the fourth 
grade, and (A) the number proportion of schools within each school 
district providing services supported through Federal funds. Further, 
States will be able to draw representative samples of schools stratified 
on variables such as geographic location, urbanism grade spans, types of 
programs, etc. The Program Reference File will also be used at the 
Federal level to draw samples for Installation of the other components of 
the total system deevibed below. 

Finally, an examination of the proposed FRF form raises several 
questions. First, it would be feasible and desirable to add one core 
dimension to questions concerning programs. The dimension proposed is the 
number of students participating In each program. This tould be eccom- 
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plishcd by having the person filling the form out use n percentage 
estimate or frequency count in the proper cell as opposed to indicating 
this with n cheek mark, os currently planned, The percentage estimate 
is viewed as the estimated percentage of students participating in each 
grade. This information would then be tied into a frequency estimate 
by using the percentage estimate in conjunction with the grade size to 
compute the number of students participating. With these figures avail- 
able for coch school a more precise population estimate would be possible 
In addition, it would provide a reliability check ond/or tit in point 
with data obtained under the Project Descriptor Inventory. 

Secondly, there arc disturbing points about the use ft two separate 
forms: l) it defeats the purpose of Belmont to reluct forms; and 2) 
it raises the spectre of incompatible data. Of the tvx>, the latter Is 
trtorc disturbing, since ultimately data abom programs trust be related to 
the information contained in the NCES Hie. Great car*-' must be taken 
to Insure this compability, This can be Accomplished by the use of the 
same USOE code number for schools in each file or inventory. 

If the above consideration can be attained then the use of the PKF 
inventory will furnish the data required for the PRF itself. In addition, 
adding the dimension of student participation will also facilitate sample 
planning ond costing. 
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listed bo 1 ow are the questions JCLS personnel would like to add to 

the Universe File Questionnaire: 

1, Population of community in v:hich is located, 

2* Population of largest community within 20 miles, 
i', Crorstabs of programs from KST‘ 1965, KDKA 1958, and 
Vocational Education Amendment 1968 by Grade Levels 
(Pro K~ 1 2 ) 

i \ , C onsol Ida t od P rogram In for mat 1 on Repor t 

Consolidated Program Information Pxcport (CPIR) represents the 
management information portion of the JCES, Historically state and 
local agencies have been forced to supply multitudinous statistical 
reports to fulfill Federal reporting requirements under the various 
legislative acts through which the government supports educational pro- 
grams. In addition to the burden imposed by filling these many duplies- 
toty and sometimes inadequate reports, there has been real difficulty 
in cross col tat fug the information contained therein to arrive at usual 
policy planning and managerial information. It is intended that the 
CPIR supplant many of these existing reports, drastically reducing the 
amount of punching and processing time, and providing a ready source 
for the information so necessary for policy and planning decisions 

The CPIR was developed by 'he program information committee of the 
Joint State/OE Task Force on Evaluation (the Belmont Group). The 
first of the CPIR forms was tried out in 1969 to collect FY 69 data, in 
a sample of 1500 nationally representative school districts. The results 
of this survey arc expected to be available by January 1970, 
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The Fiscnl 1970 Instrument will be distributed to Belmont representa- 
tives by the end of January 1970 for review and possible revision, It 
is expected that the Fiscal 70 suivcy will be conducted on a state by 
state sampling basis in nil 50 stales and will consist of about 4,000 
local school districts, so that each state will have the opportunity to 
examine and process its own information. 

In general, the 1970 Instrument differs from the 19f>9 Instrument in 
only minor ways. The desirability of introducing target groups as another 
dimension In the Instillment was reaffirmed by the Belmont Group, contingent 
Upnn the availability of ihc attribution guide, and some parts of 
Section IV (supplemental program information section) were chosen for 
elimination because of questionable use by state plans and program 
officers. The attribution guide is an adjunct to the Cl 1 IK which is to 
be developed for the purpose of attributing resources to programs in 
education, The proposed guide Is intended to provide a means of 
Attributing Information '■retained in the accounting records to the 
services Activities funding sources and target groups requested in the 
CP1R, It is hoped that a draft of this guide can be trade avail- 
able for use vith the 1970 Instrument. 

The program information conmittee of the ftelmont Group Also 
discussed the possibility of taking an active role in the development of 
a consolidated state evaluation fora (sec Section bclov) and a consoli- 
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dated application form. It was the consensus that it v.'ould be desirable 
for the Group to assume this ro.lc. No schedule was set for so doing, 
however. It will be recalled that the CPIR lr. intended to collect data 
regarding the following elementary and secondary programs ; KRFA Title I. 
II, III, V (Flow through funds), VII and VIII; Nl)FA Titles III and 
V-A; Civil Right Title IV, and the Follow through program. 

• At the present time, the present form (1969) of the CPIR consists 

of a rather bulky booklet with a manual of instructions. Part I of 
this boohlct contains identification and certification information with 
respect to the local education agency, number of pupil in public 
schools as of 1 October 1968; number of participants in Federal programs 
unspecified by public and non-public schools in regular and summer 
programs; number participating in Federal programs from various target 
populations (pupil population groups); and services and activities 
supplied in Federally aided programs by public school student level and 
non-public school. 

The next section of the CPIR is concerned with staffing and reports, 
the number of professional staff member engaged in services and activities 
supported by Federal programs as a function of the level of school type 
of program and type of pupil pcpulation group; also by number of non- 
ptofessio.nl staff members by amount of participation end by extent of 
the program villi respect to the regular scliool tern. Also in this 
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section Is personnel development: data regarding the number of staff 
members engaged at each of several professional levels; assigned to In- 
service activities and training through Federally supported programs. 

In addition, dollar amounts by types and sources of Federal funds for 
in-service training are requested. 

Part Three of the CPllt is concerned with program expenditures and 

provides amount expended by source of funds and according to several of 

the Federal source programs os function of type of service and activity 

♦ 

conducted. This section Is quite extensive end detailed in terms of the 
services nnd activities covered, <md concludes with a summary of expend- 
iture of Federal funds by instructional level. 

Fart Four is concerned with supplemental program information (it 
will be recalled that some of the 1969 CP IK content in this section is 
expected to be deleted for the 1970 Instrument) . This section gives 
tctal counts of children v:lio participated in public and non-public school 
summer school terms in the ESEA Title I Program for r hildren in Low 
Income Areas. It then gives counts by type of school and type of term 
for various grade levels participating in tin's program followed by a 
drop out information and number of program staff by school term type and 
level of program. Number of staff members receiving in-service or pro- 
service education is reported by level of professional school tern 
nirabcr of training hours received. Services end activities arc reported 
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In t.fimis in of indicating those which ore provided by typo of school 
and grndc level for both regular and summer terms. Average daily 
membership and number of children dropping out of school during the 
regular school term is given by Title 1 Public Schools and Other Public 
Schools and by grade level. 

For the ESEA Title I Migratory Worker Children Program, similar 
information similar to that for low income children is collected. 

Section C of Port. Four concerns similar information lor the Follow 
Through Program, While Section 1) of Part Four is devoted to the Title 
II, ESEA. The Title II information begins v?ith a summary of expenditures 
by categories and materials provided; indicates the number of teachers 
in the ESEA Title II program and the number of new school libraries estab- 
lished. It continues with the number of materials and expendl'ures 
for loans to children end teachers as a function of type end level of 
school and categories of materials. 

Section E of Part Four concerns MDFA Title IT I and NFAllA 1'iscal 

i 

Year 69 expenditures for instructional materials and equipment by sub- 
ject matter areas. Section F shows school membership by level, number 
of children by level, and adults end teachers receiving in-service 
training. It also shows the number and percent of children participating 
in Title III projects during Fiscal Year 69 by ethnic groups and by rural/ 

urban and socio-economic groups. Further, a table indicates type of 

« 

paid personnel by number of staff assigned to the projects as a function 
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of major program services type and level of schools and In-service training. 

It. can he seen from the above review that the information collected 
by the CPIR is extremely extensive with respect to program level and 
amount of participation in the local schools with respect to various 
programs supported by federal funds. In addition to the booklet of the 
form itself there is a manual explaining the treatment required for 
each of the categories of information and each of the tables to be 
filled out. 

Irnpl c ni ent lion Sch e dule 

The implementation of the CPIR has already begun on a sample basis. 

In the Fall of 1969 a sample of 830 school districts in all 30 states 
were chosen to be representative of all school districts that have 
ESEA Titles I or II, or NDEA I1T or V-A. In addition, all school 
districts not included in the above that had a migrant program or 
ESEA V, VII, VIII or Follov? Through were included in the sample. 

Finally, all school districts not included that had ESEA Title III program 
were asked to complete the ESEA supplemental part. This sample resulted 
in approximately 1500 school districts filling out the CPIR in toto, 
and an additional 900 filling out the Title III supplements. Data 
analysis for these administrations is underway. 

The 1970 survey with the CPIR will be conducted in all 50 states and 
A, 000 local school districts. Each state will have a sample for its 

own information and analysis. 




The following Is an item by item description of the Consolidated 

Program Information Report (CPIK) 

G ener al 

1. Name and address of Local Education Agency (LEA), 

2. LEA Representative. 

3. Stale Education Agency Official who approved report. 

4. Grade spans of elementary and secondary schools. 

Pu pib a n d Schools 

1, Public school membership hy grade level, 

2. Number pupils who participated in federally aided programs in public 
and non-public schools by grade level during regular school term 
and summer term, 

3, Number of children (ages 5-18) in school district by pupil populations 
group (handicapped, migrant, delinquent, etc.) 

4. Number of pupils in public and non- public schools by pupil population 
group who participated in federally aided programs. 

5. Number public and non-public schools which conducted federally aided 
programs for each of the pupil population groups, 

6, Number pupils in federally aided programs by type of service or 
activity during regular school term and summer tern by school level 
(Elem. , Se, , Pr. K & K) 

Staffin g 

1, Number of professional staff members (full-time and part-time) for 
federally aided programs during full re gular sch ool te rn) for each of 
the pupil population groups by type of service rendered: 

(a) Direct educative services 

(b) Supporting services 

2. Number of non-professional staff member (full-time and part-time) 
during full regular sch ool term for each of the pupil population 
groups . 

3. Number of professional staff members (full-time and part-time) for 
federally funded programs that listed only a part of the regu lar 
s chool ter m for each of the pupil population groups by type of service 
rendered: direct educative services and supporting services, 

4, Number of non-professional staff member (full-time and part- f ine) for 
programs that lasted only a part, of the regular scho ol t erm for each 
of the pupil population groups. 
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5. Number of professional staff members (full- time and part-time) for 

federally aided programs during Clio sum no r te rm for* each of the pupil 
population groups by type of service rendered: direct educative 

services and supporting services. 

6. Number of non-professional staff members (full-time and part-time) 
during summer ter:: for each of the pupil population groups. 

7. Number of staff members in district assignee to v;ork in programs 
designed to meet needs of the target group. 

8. Number of staff members who received in-service training through 
federally aided programs. 

9. Amount of funds expended for in-service training by Title. 



Program Expend it tires 

1. Amount of funds expended by Title for each of the various services 
and activities provided in federally aided programs. 

2. Amount of funds expended by instructional level (Fre K, K, 1-12, 
Post 12, dropouts, Adults). 

S upplementa l Pro gram Information 



A. 



B. 



ESEA Title I - LEA Program for Children in Low Income Areas 

1. Number of children who participated in public and non-public 
schools . 

2. Number of schools and pupils who participated by grade level 
during regular and summer school terms. 

3. Number of children who were dropouts prior to participating in 
regular or summer school program. 

4. Number of staff members by type and grade level during regular 
and summer school term. 

5. Number of staff members by type who received in-service or 
pre-service training and number of hours of such training. 

6. Number of children in public and non-public schools by grade 
level (K, 1-6, 7-12) by type of service or activity during 
regular school term. 

7# Number of children in public and non-public schools by grade 
level by type of activity or service during summer term. 

8. Average Daily membership and number of dropout from Title I 
public school and non-Title I Public Schools. 



ESEA Title I - LEA Program for children of migratory workers 

1, Number of intrastate and interstate migratory children 
participating in program. 

2. Number of participants in public and private schools during 
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regular and summer terms by grade level (pro K, K, 1-6, 7-12) 

3, Number of staff members by type and grade level during regular and 
summer terms, 

4# Number of staff members by type who received in-service and pre- 
service training and number of hours of such training, 

5, Number of children in public and non- public schools by grade level 
(K, 1-6, 7-12) by type of service or activity during regular school 
t e rm . 

6, Number of children in public and non-public schools by grade 
level (K, 1-6, 7-12) by type of service or activity during summer 
term, 

C. Follow Through Program 

1. Number of children participating in public and non-public schools. 

2, Number of participants in public and non-public schools, 
during regular and summer terms by grade level (K, 1*2,3). 

3, Number of staff members by type and grade level during regular 
and summer terms. 

4. Number of staff members by type who received in-service and pre- 
service training and number of hours of such training, 

5. Number of children in public and non-public schools by grade 
level (K, 1-3) by type of service or activity during regular 
school term. 

6, Number of children in public and non-public schools by grade 
level by type of service or activity during summer term. 

D. ESEA Title II 

■ 1. Total expenditures for textbooks, school library and other 

instructional resources from state and local funds and private 
funds (for non-public schools). 

2. Humber of teachers participating in public and non-public 
schools . 

3. Number of new public school libraries established under the 
Title in elementary and secondary schools, 

4, Number of materials and amount of funds expended by item 
category for loan to pupils and teachers at elementary and 
secondary public schools. 

5, Number of materials and amount of funds expended by item 
category of loan to pupils and teachers at elementary and 
secondary non- public schools. 

E. NDEA Title III and NFA11A, FY 69 

Expenditures by subject area for instructional materials and 
equipment (for audiovisual and other materials and equipment 
separately) . 
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F. 'ESEA Title III 

1. School membership one! number of participants by grade level 
in public and private schools. Also number of teacher who 
receive in-service training and number of adult participants. 

2. Number and percent of participants by ethnic and racial group. 

3. Rural /Urban distribution of participating children. 

4. Number of full and part-time staff member by type. 

5. Number participants by program or service in public 
schools by grade level and in non-public schools. Also 
number of adult participants and number of teacher 
receiving in-service training by program or service. 

6. Total cost by program or service. 

5 • State Evaluat ion A gency -- Managemen t Ev al nat ion Su rvey (SEA- ME S) 

The SEA Management Evaluation Survey is an Instrument designed to 
collect information which is required in order to evaluate the management 
and administration of OE projects at the state level. At the moment, 
this instrument is in the initial planning stage with a working draft 
planned for presentation at the second quarterly Belmont meeting some time 
in July 1970. The instrument itself is viewed as a questionnaire 
similar in design to the Consolidated Program Information Report. In this 
instrument, questions will be limited to details about the management 
of the program. For example, they will deal with areas such as: 

What programs do you administer? 

What types of people assist you in managing these projects? 

How many of each type, etc.? 

The information accumulated will then by analyzed and used to evaluate how 
effectively the program was managed. 

See discussion of developmental requiremnts in body of report (Page ). 
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6 . Pupil Centere d Instrument 

The purpose of the Pupil Centered Instruments will be to gather 
information on school districts, individual schools, teachers, and pupils. 
Part of this instrument will be aimed at determining the extent to which 
individual pupils participate in the various programs and activities 
described through the Project Descriptor. In addition, the instrument 
will allow for the determination of the background and characteristics 
of the individual pupils participating in the programs. 

The Pupil Centered Instrument will consist of four basic components: 
School District Questionnaire 
Principal Questionnaire 
??acher Questionnaire 
Pupil Questionnaire 

These questionnaires are based upon revisions of the 1969 Survey on 
Compensatory Education but the data collected by the 1970 instruments 
will be applicable to the evaluation of several programs in addition to 
ESEA Title I. The 1970 instalment 'for elementary schools will be imple- 
mented in the Spring of 1970 in a nationally representative sample of 
school districts from all fifty states. As with the 1969 Survey, the 
grades sampled will be grades 2, A, and 6. Teachers will be requested to 
complete the pupil questionnaires for no more than A of their students. 

The secondary school instrument has. not been developed and will not be 
used in this year's survey. 
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The Development Task Force lor the Tupil Centered Instruments antici- 
pates t: ! 1 a t new questions will be developed for the 1970 Instruments. 
Questions will be developed to provide da La on guidance, counseling, 
and occupational orientation programs for elementary school pupils. Since 
a complete description of available programs and services is desired, 
new questions will be developed to gauge the appropriateness, adequacy, 
accessibility, and use of the facilities, materials, and equipment that 
are available to districts, schools, and pupils. Finally, because 
of the complexities of local program administration, data will not be 
requested according to the Federal funding title providing the services. 

Only two questions of a general nature will be included in the School 
District and Principal Questionnaires relating services to funding 
source . 

The following is an item by item description of the four questionnaires. 
1969 Pupil Centered Instrument - School District Questionnaire 

r 

General Information 

1. Number of public schools in district by grade span. 

2. Annual salary for starting elementary school teacher, 

3. Maximum salary for elementary school teacher. 

Test Data Info rmation 

1. Names of achievement test batteries, levels and forms of pre-tests and 
post- :ests and dates of test administrations in Grade 2, A, 6, 

2. Most recent dates the following types of tests were administered in 

Grade K-12: reading readiness 
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Arithmetic or Hath 
Language 

Foreign Language 
Intelligence 1 

3. Type of norm groups used to compute scores for Grades 2, 4, 6. 

4. Conversion tables for local norms (if used). 

ESEA Ti tle JL_ Info rmation * 

1, Amount of funds approved for use in FY69 (Title 1). 

2, Number of school pupils who received services funded by ESEA, Title 
1 by grade, 

3, Number of public schools providing programs and services funded by 
ESEA, Title I by grade. 

4, Basis used for allocating funds from ESEA Title I. 

*l f or the 1970 questionnaire, this section will be revised where 
appropriate. 

Pa ren t Invo l vement In f o r mnt ion 

1, Method of securing parent involvement in ESEA, Title 1 program. 

2, Total number of ESEA, Title I Citizens' Advisory Committees active 
in district .** 

3, Did State Dept, of Education give assistance or advice in creating 
Citizens 1 Advisory Committees?** 

4# Concerns of Citizens Advisory Committees. 

5, Duties of Citizens' Advisory Committees.** 

6, Primary methods of selection of different types of persons serving 
on the Citizens' Advisory Committees.** 

** These items will be eliminated from 1970 questionnaire. 

Person n el Tra i.ning Informa t ion* ** 

1. Amount of ESEA Title X funds expended for support of in-serv:ce 
training programs. 

2. Number of personnel by type who have participated in in-service 
training program funded by ESEA Title I. 

3. Have classroom teachers participated in in-service training programs 
funded by ESFA Title I which incorporated joint training v;ith teacher 
aides, other professional personnel of parents? 

4. Main objectives in In-Service training program supported by ESEA 
Title I for the different types of school personnel. 

5. Subject matter for ESEA Title I funded In-Scivice Training Programs. 

6. Grade levels of instructional techniques covered by the ESEA Title I 
funded In-Service Training Programs. 

***For 1970, this section will be modified to include all programs. 
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6. Do Lx. hers use instructional resources from reference centers out- 
side the school. 

7* Number of classrooms for grades K~6 with following equipment: 

(a) Projection screen. 

(b) Light control for projection. 

(c) In* room terminals for T.V. antenna, 

(d) In- room connections for closed-circuit T.V. 

(o) Independent study stations. 

(f) Electrical outlets, 

8. Does school provide material or equipment to acquaint pupils 
with Vocational Education Programs? 

9. Copyright date of most frequently used texts in math and reading 
in regular school program in grades 2,4, and 6. 

S t: udent B ody Da scri p! i o n _ 

1. Percent of pupils from minority groups. 

2. Percent of pupils whose families are on a public welfare program. 

3. Percent of pupils whose parents did not complete 8th Grade. 

4. Are standardized reading achievement test results available? 

5. Percent of 6th grade pupils who are at least one grade level but 

less than two grade levels bclov; national norms in reading achievement. 

6. Percent of 6th grade pupils who are at least two grade levels or more 
below national norms in reading achievement, 

1969 Pupil Centered Instrument - Teachers Questionnaire 

Teach e r s B ackgroun d Information 

1. Grade taught. 

2. Sex. 

3. Years of full-time teaching experience. 

4. Years of teaching in this particular school. 

5. Reside within attendance area of school? 

6. Member of minority group? 

7. Number of days of absence when school v?as in session* 

8. Reason for teaching at this particular school. 

9. Training in teaching academically disadvantaged children. 

10. Hours of participation in formal in-service training concerned 
with academically disadvantaged children. 

11. Participation in in-service training program concerned with 
vocation guidance or occupational information for elementary pupils. 

12. Indicate whether any of the services listed below has been provided 
and whether they have * een funded by ESKA Title I. 
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(n) Classroom aides. 

(b) Consultants, 

(c) Tuition fees for college or summer institute. 

(d) Professional travel. 

Class Cliarac tori s tics and Orgnnizn tiori 

1. Number of pupils In class from families on welfare. 

2. Primary source of information used to respond to above item, 

3. Percent of pupils in class that arc members of minority groups. 

4. Percent of pupils in class that are performing belov; grade, level 
in reading. 

5. Number of pupils in .class on each of two dates. 

6. Number of pupils entering class bctvjecn two dates. 

7. Number of pupils leaving class between two dates. 

8. Number of teachers that have held your teaching assignment, with your 1 ' 
class for at least two consecutive weeks. 

9. Percent of pupils participating in programs for the academically 

disadvantaged in the following subjects: math, reading, language, 

other academic subjects. 

10. Mien do students participate in program for the academically 
disadvantaged in math, reading, language and other academic subjects: 
before school, after school, weekends, or during regular school clay. 

11. Description of classroom organization: specialist teachers, 

teaching aides, team teaching, non-graded classes, departmentalized 
grouping by ability, etc. 

Tea ching Metho d 

1. Description of instructional program: regular, academic disadvantaged , 

enrichment. 

2. Indicate subject areas and programs which you regularly teach: 
reading, math, langunge/enrichment , programs for the acadcmcially 
disadvantaged . 

Pro g ram of Instruction in Yo ur Cl ass 

1. Dow are pupils grouped for instruction in math, reading, language? 

2. Number of minutes per Instructional period in math, reading, and 
language. 

3. Number of instructional periods per week in math, reading, language, 

4. Number of weeks of instruction in math, vending, language. 

5. Description of approach to material, presentation in math, reading, 

and language: topic, subject matter, unit, skills, or activity 

centered. 
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6. Description of curriculum approaches or concepts in math, rending 
and language . 

7. Types of responses oxpecLed from pupils in rending, math, and language 
after some material has been presented. 

8. Educational objectives emphasized in reading, math, and language. 

Teac h e.r Conc erns* 

1, Is class progress hampered by individual differences of pupils? 

2, Discipline problems. 

3, Adequacy of equipment and materials available. 

4, Appropriateness of equipment and materials 

5, Do you have a part in chostng the classroom curricula? 

6, Are programs for the academically disadvantaged generally v?orthwhile? 

7, Description of classroom activities: 

(a) Giving mild or strong approved or disapproval, of pupil 
actions . 

(b) Extending, modifying, or elaborating upon pupil ideas 

(c) Asking questions of pupils. 

(d) Giving direction to pupils. 

(c) Li stoning to pupils. 

8, Teacher opinions on various matters of teaching,** 

* This section will be reduced in the 1970 questionnaire. 

** This item will bo eliminated in the 1970 questionnaire. 

1969 Pupil Centered Instrument - Pupil Questionnaire. 

General Information 



1. 


Fupi I 1 s grade level. 




2. 


Pupil's sex. 




3. 


Pupil's month and year of birth. 




4. 


Month of school year that pupil started 


class . 


5. 


Number of days of absence. 




6. 


Uere absences due to illness? 




7. 


Since first grade, how many schools has 


this pupil attended? 


8. 


Is pupil: 





(a) from an institution for neglected children, 
(b> from an institution for delinquent children, 
(c) from an agricultural migrant family. 



9. School experience before first grade. 
Pupil Backgroun d In f ormation 
1, Member of minority group. 
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2. Language oilier than English primarily spoken In home* 

3. Occupation of Father. 

4. Pupil's family on public welfare? 

5. Estimate of family income* 

6. Source of information for family income. 

7. Relationship of head of pupil's household to pupil. 

8. Is pupil's household head employed? 

9. Educational level of head cf household and Mother. 

10. Is Mother employed? 

11. Description of area near pupil's home: residential, rural, residential 

and cornmorical 

12. Considering present attitude, how far v?ill pupil go in school? 

13. Considering ability, how far will pupil go in school? 

14. Educational aspiration of parents for pupil. 

15* Have you bad any communication with parent during school year? 

16. Nature of communication, 

17. Reading level of pupil. 

18. According to pupil's critical needs, what program do you recommend 
for him? 

Acad emic Pro gram E ar tlc.lpn t i o n Info r ing t ion 

1. Does pupil participate in regular programs in math, reading, language? 

2. Number of programs for academically disadvantaged or enrichment programs 
in which pupil participates in math, reading, language. 

3. On what basis was pupil placed in these programs. 

4. Average size of instructional group for each program. 

5. . Number of minutes per instructional period for each program. 

6. Number of instructional periods per week for each program. 

7. Number of weeks for each program. 

8. Curriculum approaches or concepts for each program. 

Ancillary Program Par tic ip at ion Information 

1, Types of cultural enrichment programs in which pupil participated. 

2. Total number of hours in nil cultural enrichment programs* 

3* Has pupil participated in a health seivicc program in school? 

4. Which of following health services were provided: 

(a) Physical, dental, or eye examinations 

(b) Treatment or therapy. 

(c) Other. 

5. Has pupil participated in programs for treating social, emotional, or 
disciplinary problems? 

6. Does pupil receive free or reduced price food in school? 

7* Ibw much Ins pupil benefited from ancillary service program? 
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S ummer P ro g nuns 

1. Did pupil part) ci pale in reading, math, or language summer program? 

2, Length of program in weeks and hours per week, 

3. Cultural enrichment proginn during summer - type. 

4, Number of hours spent in cultural enrichment program during sunencr. 



lhipi) beh avio r 

1, Changes in pupil's academic performance* nnd behavior, 

2, Percent of time spent in: academic work, other learning activities, 

disruptive behavior, other non- constructive behavior, 

3, Interests in math, reading, and language. 

Test P erfo rmancc 

1. Has pupil taken c standardised achievement test since Sept. 1967; 
between Scpl . 1967 and l)ec. 19687 

2. Month and year of pre-test administration. 

3. .Name, level, and form of test. 

4. Have pupil's scores been reported to school? 

5. Hire- test scores of pupil: rending, arithmetic, language, 

6. Veto lasts part of regular school program or ndmini stored in relation 
to compensatory education? 

7. Has pupil taken a standardzied achievement test since Jan. 1969? 

8. Month and year of test administration. 

9. Name, level, and form of test. 

10, Have pupils scores been reported to school? 

11, • Host-test scores for pupil: reading, arithmetic, language. 

12, kero tests part of regular scliool program or administered in rela- 
tion to compensatory education. 

Supp lementary Pu pil It ^forma tion* 

1. People vho live vith this pupil in house or apartment.** 

2. Have parents talked to pupil about college? 

3. Have parents discussed vith pupil occupational cltoicc? 

4. kliat materials does pupi 1 rend outside of school?** 

5. I>>es pupil belong to Hoy or Girl Scouts, etc.? 

6. Hoes pupil take art or icusic lessons outside of school? 

7. before first grade did anyone teach pupil alphabet, etc.? 

8. Vba t vould make parents happier? !*>ing veil in school work 
or acting very badly? 

9. ttbat vouid'tnake parents unhappler? Poing badly in school work or 
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behaving very badly. 

10. Source of information of above. 

* This section will be reduced considerably for the 1970 questionnaire 
** Ihesc ileitis v;ill be eliminated from the 1970 questionnaire* 

7 . Pro j ect De scriptor . Jjn r% 1 .rv i a n t 

The Pro gram Reference File for any selected school, will contain 
information on the general types of programs at different grade levels, ns 
well ns the funding sources for these' programs. By drawing a sample, or 
samples, from the Program Reference File and by administering a Project 
Descriptor Instrument to the schools in the sample, specific descriptions 
of projects and activities operating as components in any general program 
can be developed. 

Hove importantly, this will enable these projects or activities to be 
related to data gathered through a Inpil Centered Instrument. For example, 
inferences about what kinds of activities produce vV.. kinds of results 
for what kinds of children will be possible. 

The projector descriptor instrument will he designed to gather the 

i 

following types of information: 

1. Type of project or activity; c.g., remedial math 

2. Source of funding for project or activity 

3. Duration of project or activity 

4. Number of participants and background characteristics of 
participants 

3. Organization for instruction, including teaching mctliods 
employed 
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6. Facilities, equipment, .and Materials utilised. 

The Project Descriptor Instrument is being developed under contract 

to Iklucr.tional Testing Service, Princeton, New Jersey. Hu* project 

director has stated that the object of the Project Descriptor Instrument 

is to obLain data sufficiently detailed to allow differentiation among 

educational treatments. The instrument, therefore, contains ileins 

descriptive of target groups, plant equipment, materials, personnel, and 

\ 

services. Information is to be reported about each project whore a 
project is basically defined ns a collection of people, goods and 
services for which application was made and funds received. (It should 
be noted that some projects arc difficult to define and that the Projector 
Descriptor Instrument provides a series of decision rules for deciding 
when a project should be subdivided for reporting). 

The Project Descriptor Instrument has been built to measure the 
categories in a taxonomy of project program activities which vas developed 
as a part of the FfS contract. The specific questions in the ProjeeL 
Descriptor Instrument are designed to cover the various categories of the 
taxonomy. For this reason, the taxonomy will be summaritd bclov;. 

There are sixteen major categories in the taxonomy: 

I . Identification of Pro joe t _ P rogram Activ ity 

lhis category contains Lhe title or description of the project, the 
control number, information about the project director, and the ndrainistra 
Lion of the project . 




II , fundin g 



This category contains information about the source of federal funds 
for the project and the status of the request for federal funds in terms of 
requested) approved or received, and the fiscal years in question. It 
covers nil of the sources of federal funds discussed earlier with the 
exception of Civil Rights Title IV. This category goes on the cover the 
relevance of specified federal funds, state regulations and priorities 
on the distribution of funds, and sources of any other Federal funds 
for this project program or activity. 

III. liuration o f the Project 

IV. l ocation of th e Pro ject 

This category is expressed in terms of geographic or administrative 
unite, economic characteristics, location, population and ethnic composi- 
tion. 

V . Project Related Needs of the Impact Area 

This category covers type, of need ranging from improved or expanded 
educational technology to improved plant or facilities; avrarcncss of 
needs; and drav.’s on students, teachers, parents, school systems, and 
communities as sources. 

VI. Project Pelf ni.fation 

This category covers the specification of the use of the support 
vith respect to planning, Implementation of existing plans in toms 



of construction, rental, purchase, rtmc.leling, and development, etc. It 
also covers personnel, services r.nd scope of the project. 
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VII. T arget Grou ps 

Target groups are defined in sub-categories such as pupils, broken 
down to public and non-public, ethnic composition, so:, socio-economic 
status, special characteristics, eligibility and procedures for selection; 
local educational agency as a target group including public and non-public 
school, grade levels, ethnic composition, socio-economic status selection; 
and, finally, groups influential in setting educational goals such as 
state school board members, state education agencies, community leaders, 
pa rent- community advisors, etc, 

I 

VIII. Physical 1-lant 

School buildings and property! types of facilities, location in 
respect to impact area, availability, and adequacy; non-school building 
and property: types of facility, location, availability, and adequacy. 

IX. Eqnl pment 

Typo of equipment and adequacy. 

X . Materials an d Supplies 

* 

Type, ranging over art to driver education adequacy. 

XI. Personnel 

Professional Instructional personnel! Types, adequacy; non-professional 
instructional personnel: Types, adequacy. 

XI I. Noi;r instructional Services 

For pupils, for local, or state agencies; objectives of such service, 
and duration of time of such services. Services covered In these categories 
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range from library, counseling, and welfare to statistical data processing, 
surveying needs, etc. 

XIII. Instructional Services Initialed, Contin u ed or Expanded 

A. For pupils: types of services, objective of the services, 

duration of the services; B. For state local, education agencies: types 

of services, objective of the services, and duration; C. Methodology: 
for student groupings and techniques, approaches, classroom activities, 
teacher-student interactions. 

XIV. Planning Methodology 

A. By participants; B. By qualification of the leader; C. Sharing 
of Responsibility; D. Planning of the Product. 

XV. Evaluation 

A. Data from individuals and from teachers; various kinds of tests 
and measures, interviews, achievement, etc.; From documents: Design of 

evaluation, the evaluators, their affiliations, qualifications, amount of 
time, involvement of planning and products. 

XVI. Dissemination 

Targets: Local education agencies, teachers, students school board, 

etc.; cowaunlty targets, state education agency targets, professional 
tArgets. Form of dissemination: Written or oral, and sub-categories: 

author of dissemination material project director, evaluator, instructional 
staff, others. 
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The Project Descriptor Instrument as it stands in its present, pre- 
test form is a very long and cumbersome instrument. However, it should 
be noted that there are to be several parts to the Project Descriptor 
Instrument one each to be filled out by the school distrLct project 
coordinator or director, the principal, and the teacher. Also, the 
Project Descriptor Instrument was not originally intended to contain 
outcome data. However, the present plan involves the development of 
Additional outcome questions for the Project Descriptor Instrument so that 
In the event that Pupil Centered Instruments do not produce outcome 
data for some reason, there will be some outcome data available through 
this source. Tentatively outcome questions might included information 
about numbers of pupils in various categories, attendance and perhaps an 
achievement test score... etc. 

Pretest of Project Description Taxonomy 

The Project Descriptor Taxonomy was pre-tested during the month of 
November in the Belmont states of Texas, Washington, Connecticut, North 
Dakota and Maryland. Zach of the five states were requested to provide 
OE with a list of 80 federal projects operating in their states. These 
80 federal projects were chosen on the basis of the following criteria: 

1. Level of operation-elementary or secondary requiring roughly 
projects from each level. 

2. Grade of operation--so«e projects for all grades should be 



included 
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3. Subject of pro ject--either academic or special service areas 
requiring roughly 40 projects in each area and including 

at least tv.x> or three staff development projects 

4. Urbanism of school districts offering proJects--either urban, 
suburban, or rural. 

Of the 80 projects for which descriptions were to be provided, only 
20-30-were to be pretested. OB made the contacts with the participating 
project directors and school districts. ETS mailed the project descriptor 
taxonomy for the follow-up with the participants. 

Project descriptor; instrument data analyses will provide detailed 
investigation of th;? a f rvices provided through each Federal legislative 
title. For example, reports will indicate the structure, methodology, 
materials, equipment and personnel used in projects in compensatory read- 
ing in secondary schools. The frequency of use of various techniques 
will be reported for the 1969-70 school year and the relative frequency 
of use of various techniques will be reported for the techniques, facilities, 
materials, etc. Because of project descriptor will include information on 

V ^ - „ ■ * - ^ ^ ^ •'****.*. ***' J*- Vt* ^ 

the types of pupils participating in projects, analyses will report the kinds 

* « 

of services being rendered various kinds of pupils within states. When the 
entire system is implemented, the relative effectiveness of various kinds 
of services for various kinds of pupils will be reported. The initial 
year reports will include analyses of the character of secondary school 
projects by subject area and by source of funding within States and for 
the 20 states in aggregate. Program managers in State agencies wilt thus 
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have comprehensive pictures of the kinds of services each of the Federal 
programs is supporting in their States. 

The following is ar. item by item description of the Project Descriptor 
Questionnaire: 

General 



1* Title of Project 

2. Project SDE number 

3. USOE contract number 

4. Name and address of individual with' major responsibility for project 
Also, Phone number, Job Title, and affiliation 

5. Name of Agency administering project 

FundlnR 

6. Amount of federal funds requested for current FY by Title 

7. Amount of federal funds received for current FY by Title 

8. Amount of federal funds requested for all prior years by Title 

9. Amount of federal funds received for all prior years by Title 

10. Total time period for project supported by Federal funds 

11. Anticipated amount of time that project will continue to receive 
federal funds 

12. Total project expenditure through end of current FV 

13. . Were fund9 received on time? If not, any problems? 

14. Amount of federal funds received for current FY by Funding Source or 
Agency 

15. Amount of federal funds received for all prior years by Funding 
Source or Agency 

16. Amount of State funds received during current FY 

17. Amount of State funds received during all prior years 

i 18. Amount of funds other than federal and state received for current 
FY by type of source. 

19. Amount of funds other than federal and state for all prior years by 
type of source 

20. Total project titae period to date supported by any source 

21. Anticipated future period of funding from any source 

Impact Area 

22. Unit of impact area • 




state, county, school, etc. 
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23. Average income of impact area 

24. Unemployment rate 

25. Per Pupil Expenditure for all public achools serving area 

26. Population 

27. Minority group percentages 

28. Urbanism of area 

29. List of needs. Rate degree to which need characterized impact area 
before inception of project< Rate relevance of project to need. 

Needs related to educational problems and funds. 

30. Considering the above needs, identify agencies or persons who informed 
project executive group about urgency of these needs. 

Project Planning 

31. Who participated in planning? (groups or types of people) 

32. Qualifications of people in charge of planning 

33. Amount of sharing of responsibility among planners 

34. Percent of total project staff man-hours devoted to planning 

35. Projects of planning 

• I 

Dissemination of Project Information 

36. What groups received project information? 

37. Form of project information (newspapers, TV, public lectures, etc.) 

38. Authors of project information 

Evaluatio n 

39. Organization affiliation of person doing evaluation 

40. Professional background of evaluator 

41. Mas evaluator Involved in project planning? 

42. Percent of project man-hours devoted to evaluation 

43. &pcrimental design used 

44. Evaluation product 

45. Data used 

Sub-Pro ject 

46. Number of Sub-projects 

47. Crosstab of types of goods, services, and people by sub-project 

The remaining portion of the questionnaire is completed separately for each 
sub-project. 
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General 



l. Name, address, phone number, job title, and alfiliation of person 
responsible for sJb-projcct. 



Target Group 

2. Number of persons In tarp >t group, by grade level for public and non- 
public schools separately. 

3. Ethnic compositions of target group 

4. Number of males and females in target group 

5. Number of person In target grep whose families are on welfare 

6. Categorization of target group by Its particular social problem. 
Categorization of school personnel by job title. 

7. How was eligibility for particip? tion in project determined? 

8. How were target group members selected? 

Adult Community Members of Target Croup 

9. Number of adult community members of target group by minority group 

10. Number of adult community member on public welfare 

11. How were adult community members selected? 

12. Adult conmunity groups that are part of target groups (School Board, 
business leaders, etc.) 



Facilities 

13. Wiat facilities and buildings do funds pay for? 

14. • Do physical plant and facilities that were paid for by funds belong 

to public school? 

15. Vhat public school facilities are involved in the project and how 
much money was spent on ecch? 

16. location of public school facilities in relation to impact area 

17. Distance and availability of puollc school facilities to target group 

18. Adequacy of public school facilities 

19. Vhat non-public school facilities are involved in the project and 
how much money was spent on each? 

20. Distance and availability of non-public school facilities 

21. Adequacy of non-public school facilities 



nd Materials 



22. Docs project receive funds for planning purchase, purchase, or plan- 
ning the use of equipment and materials? 
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23. What types of equipment and materials are Involved and how much money 
was spent on each? 

24. Adequacy of equipment and materials 
Objectives 

25. Objectives of services provided in sub-project 
Personnel 



26. Does project receive funds for planning and/or use of instructional 
and non- instructional personnel? 

27. Number of non-instructional personnel by type (planned and used) 

28. Adequacy of non-instructional personnel 

29. Number of professional instructional personnel by type (planned and 
used) 

30. Adequacy of professional instructional personnel 

31. Nbmber of non-prefessional instructional personnel by type (planned 
and used) 

32. Adequacy of non-prefessional instructional personnel. 

Instructional Activities 

33. Does project provide funds for planning and/or implementing instruc- 
tional activities for pupils or school personnel? 

34. Types of instructional activities for pupils. 

35. Duration of instructional activities for pupils 

36. When does instructional activities for pupils take place? - in school, 
after school, weekend, etc. 

37. Objectives of Instructional activities 

38. iypes of instructional activities for school personnel 

39. Duration of instructional activities for school personnel 

40. When do instructional activities for school personnel take place? - 
in school, after school, weekends, etc. 

Non- Instructional Services 



41. Types of non-instructional services for school personnel. Amount 
of funds for earh w these services 

42. Types of non-instructional .ervices for pupils. Amount of funds 
for each of these services. 

43. Wren are non-instructional services for pupils available? In school, 
after school, etc. 
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Instructors 

44. Number of instructors used in sub-project. 

The following items are to be completed by instructors: 
Methods of Instruction 



1. How are students grouped for instructional activities? 

2. Instructional approaches (Classroom, independent, seminar, tutoring, 

P.I., etc.) 

3» Teaching techniques 

4. Instructional approaches (tapes, subject matter, unit, skills, activity 
centered, etc.) 

5. Time devoted to teacher-, student-, and teaching-guide suggested 
activities 

6. Time devoted to textbook 

7. Percent of class assignments given to whole class (same assignment 
for all students) as opposed to individual assignments. 

8. Time devoted to supplementary reading outside of main textbook 

9. Types of teacher - pupil interaction (teacher leads, teacher demonstrates, 
teacher directs, student initiate, student questions, etc.) 

8. The AIR Guide 

While the Project Descriptor Instrument is intended to be used in 
all of the Belmont states to supply project and program description 
material, an additional instrument or device, the AIR Guide for preparing 

i 

Evaluation Reports of Educational Programs, is intended to supplement the 
Project Descriptor Instrument. The AIR Guide was developed in response 
to the need identified by the Belmont group to supply some help to local 
evaluation personnel in their efforts to produce evaluation iniormation 
useful in reporting to the state and Federal agencies. The kinds of 
reports produced with t i.n use of this Guide are expected to ba narrative 
reports describing the evaluation efforts at the local level. They 
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should, however, be complementary to the information described In the 
project descriptor. 

It is of course a common requirement these days for the local 
education agency to be required to produce a report covering its own 
evaluation of on education program. The AIR Guide is intended to help 
the local evaluation team to decide what to include and how to report 
it. In the first of the sections of the Guide there are contained 
suggestions about the material which should be included to describe 
the setting (situation, context, background) of the program being 
evaluated. The second section is concerned with the description of the 
program itself (and it ic here that the narrative report produced is to 
be complementary to the information supplied by the Project Descriptor 
Instrument, In fact it has been proposed that the Descriptor Taxonomy 
be included as an Appendix to the Guide). Thirdly the Guide provides 
a discussion of the methods of reporting evidence of change brought 
about by the program in question; and finally, the format and nature 
of recommendations which may be made on the basis of the evidence 
discussed. 

The Guide attempts to be all things to all people, and freely points 

out that some of the sample questions, some of the organizing framework, 

* 

and some of the example.? do not pertain to all the programs. But it 
encourages the local evaluation personnel to select from amongst those 
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aspects of the presentation which are most relevant. In general, each 
section in tlu: Guide includes framework questions referring to matters 
which might be important in writing the report. These are generally 
followed by explanations, examples, definitions end the like. Finally, 
model answers or model narrative statements are offered as a guide to 
handling each of these sections. 

In addition to its ostensible use as a guide and aide to writing 
local evaluation reports, it is quite likely that the AIR Guide could 
be used as the framework of a training Instrument in which some local 
personnel may be taught some of the key and important characteristics to 
be concerned with in designing and carrying out evaluation work. 

9 . Common Status Measu res 

In order to describe the basic level of the pupils participating in 
the programs, a portion of the Pupil/Project-Centered Information System 
was designated the Common Status Measures . The common status measures 
include two basic scales, or.e entitled Basic Verbal Status (literacy) 
and the other entitled Occupational Cognizance (Occupational Awareness). 
The Occupational Cognizance scale is designed to assess the 4th and 1th 
grade student's knowledge of occupations and his expectations with respect 
to occupations and educational experiences. The plan calls for the 
development of several tests forms each having 12 items, 10 items of the 
four choice type covering questions corcerninf education o.. training 
required for specified occupations, the nature of work involved in 
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specified occupations, recognition of other occupations related to a 
specific occupation, and recognition of field of work corresponding to 
e specified occupation. In addition to items on these topics each 
form will contain one question each concerning occupational expectations 
and educational expectations. 

The Basic Verbal Status scale is also intended to apply to fourth and 
eleventh grade students and each test form will have again 12 questions. Ba 
sic skills being measured are vocabulary recognition and reading compre- 
hension rather than concept formation, reasoning analogies and the like. 

The Vocabulary section of the test is concerned with synonyms and cate- 
gory relationships and the distractors are chosen according to principles 
of selecting mis-leads in the following areas: 1) paradigmatic association 

2) syntagmatic association, 3) spelling similarity, 4) phonological 
association, 5) phonological similarity, and 6) opposities. In the 
category relationships the student is asked to find a work from among the 

* 

options that does not belong in the same category as the other three words. 

The reading comprehension test is of a form in which the student 
Is asked to read a relatively short passage and to choose from among four 
sentences the one sentence which makes a clear re-statement about some- 
thing that appeared in the package. The test does not emphasize generali- 
zations or inferences. 
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It might be noted in passing that the length proposed fo. - these 
tests does not suggest that they wltl have a particularly high degree 
of reliability. Perusal of the item pool suggests that the items are 
reasonah .y appropriate examples ox the types described above. 

Items included in Lhe Common Status Measures are as follows: 

Basic Verba l Statu s (12 items) 

1. Vocabulary - synonyms and semantic categories 

2. Reading comprehension. 

Occupational Cognizance (12 items) 

1. Knowledge of occupation. 

(a) Education or training required for specified occupation. 

(b) Nature of work involved in specified occupations. 

(c) Recognition of other occupations related to specified 
occupation. 

(d) Recognition of the field of work corresponding to a 
specified occupation. 

2, Occupational expectations - choose from list the occupation he is 
most likely to have as an adult or the one that is most like the one 
he expects to follov;. 

3* Educational expectations - the student is asked to indicate his 
educational plans upon leaving high school. 



